2026-09-30 15:52:16,339 [ 93316 ] INFO : ClickHouse root is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse (runner:42, check_args_and_update_paths) 2026-09-30 15:52:16,339 [ 93316 ] INFO : Cases dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:86, check_args_and_update_paths) 2026-09-30 15:52:16,339 [ 93316 ] INFO : utils dir is not set. Will use /home/ubuntu/_work/ClickHouse/ClickHouse/utils (runner:97, check_args_and_update_paths) 2026-09-30 15:52:16,339 [ 93316 ] INFO : base_configs_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/programs/server, binary: /home/ubuntu/_work/_temp/test/build/clickhouse, cases_dir: /home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration (runner:99, check_args_and_update_paths) clickhouse_integration_tests_volume Running pytest container as: 'docker run --rm --name clickhouse_integration_tests_fuqawq --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_aggregation_memory_efficient/test.py::test_remote test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True]' test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database]' 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid]' test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False]' 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True]' test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_cgroup_limit/test.py::test_cgroup_cpu_limit 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node]' 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries]' test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded]' test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True]' test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default test_config_xml_yaml_mix/test.py::test_extra_yaml_mix test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_default_database_on_cluster/test.py::test_default_database_on_cluster test_detached_parts_metrics/test.py::test_numbers_of_detached_parts 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed]' test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml test_disabled_access_control_improvements/test_row_policy.py::test_introspection test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies 'test_disk_over_web_server/test.py::test_cache[node2]' test_disk_over_web_server/test.py::test_config_reload test_disk_over_web_server/test.py::test_incorrect_usage test_disk_over_web_server/test.py::test_page_cache test_disk_over_web_server/test.py::test_replicated_database test_disk_over_web_server/test.py::test_unavailable_server 'test_disk_over_web_server/test.py::test_usage[node2]' test_disks_app_func/test.py::test_disks_app_func_cp test_disks_app_func/test.py::test_disks_app_func_ld test_disks_app_func/test.py::test_disks_app_func_ln test_disks_app_func/test.py::test_disks_app_func_ls test_disks_app_func/test.py::test_disks_app_func_mv -vvv" altinityinfra/integration-tests-runner:cf95077f4609 '. Start tests ============================= test session starts ============================== platform linux -- Python 3.10.12, pytest-7.4.4, pluggy-1.5.0 -- /usr/bin/python3 cachedir: .pytest_cache rootdir: /ClickHouse/tests/integration configfile: pytest.ini plugins: repeat-0.9.4, order-1.0.1, random-0.2, timeout-2.2.0, reportlog-0.4.0, xdist-3.5.0 timeout: 900.0s timeout method: signal timeout func_only: False created: 10/10 workers 10 workers [100 items] scheduling tests via LoadFileScheduling test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] test_ddl_alter_query/test.py::test_alter test_buffer_profile/test.py::test_buffer_profile test_disks_app_func/test.py::test_disks_app_func_cp test_disk_over_web_server/test.py::test_cache[node2] test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] [gw9] [ 1%] PASSED test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile [gw9] [ 2%] PASSED test_buffer_profile/test.py::test_default_profile test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async [gw3] [ 3%] PASSED test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management [gw3] [ 4%] PASSED test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml [gw3] [ 5%] PASSED test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml test_disabled_access_control_improvements/test_row_policy.py::test_introspection [gw3] [ 6%] PASSED test_disabled_access_control_improvements/test_row_policy.py::test_introspection test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies [gw3] [ 7%] PASSED test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies test_cleanup_after_start/test.py::test_old_dirs_cleanup [gw6] [ 8%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] [gw6] [ 9%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] [gw1] [ 10%] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] [gw8] [ 11%] PASSED test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] [gw9] [ 12%] PASSED test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async [gw1] [ 13%] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] [gw8] [ 14%] PASSED test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] [gw1] [ 15%] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts [gw1] [ 16%] PASSED test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes [gw1] [ 17%] PASSED test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table [gw1] [ 18%] PASSED test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication [gw1] [ 19%] PASSED test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host [gw6] [ 20%] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] [gw1] [ 21%] PASSED test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_default_database_on_cluster/test.py::test_default_database_on_cluster [gw1] [ 22%] PASSED test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation [gw3] [ 23%] PASSED test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default [gw5] [ 24%] PASSED test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async [gw1] [ 25%] PASSED test_backup_restore_on_cluster/test.py::test_mutation [gw5] [ 26%] PASSED test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container test_backup_restore_on_cluster/test.py::test_projection [gw5] [ 27%] PASSED test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container [gw1] [ 28%] PASSED test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database [gw1] [ 29%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async [gw1] [ 30%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts [gw1] [ 31%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] [gw1] [ 32%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] [gw3] [ 33%] PASSED test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default [gw1] [ 34%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] test_backup_restore_on_cluster/test.py::test_replicated_table test_config_xml_yaml_mix/test.py::test_extra_yaml_mix [gw1] [ 35%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster [gw1] [ 36%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster [gw1] [ 37%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def [gw7] [ 38%] PASSED test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible [gw1] [ 39%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def [gw7] [ 40%] PASSED test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_aggregation_memory_efficient/test.py::test_remote [gw1] [ 41%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge [gw1] [ 42%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath [gw1] [ 43%] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges [gw1] [ 44%] PASSED test_backup_restore_on_cluster/test.py::test_required_privileges test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] [gw6] [ 45%] PASSED test_default_database_on_cluster/test.py::test_default_database_on_cluster [gw3] [ 46%] PASSED test_config_xml_yaml_mix/test.py::test_extra_yaml_mix [gw2] [ 47%] PASSED test_disk_over_web_server/test.py::test_cache[node2] test_disk_over_web_server/test.py::test_config_reload test_detached_parts_metrics/test.py::test_numbers_of_detached_parts [gw2] [ 48%] PASSED test_disk_over_web_server/test.py::test_config_reload test_disk_over_web_server/test.py::test_incorrect_usage [gw2] [ 49%] PASSED test_disk_over_web_server/test.py::test_incorrect_usage test_disk_over_web_server/test.py::test_page_cache [gw1] [ 50%] PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] [gw9] [ 51%] PASSED test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] [gw8] [ 52%] PASSED test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability test_cgroup_limit/test.py::test_cgroup_cpu_limit [gw1] [ 53%] PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] test_backup_restore_on_cluster/test.py::test_system_functions [gw2] [ 54%] PASSED test_disk_over_web_server/test.py::test_page_cache test_disk_over_web_server/test.py::test_replicated_database [gw2] [ 55%] PASSED test_disk_over_web_server/test.py::test_replicated_database test_disk_over_web_server/test.py::test_unavailable_server [gw1] [ 56%] PASSED test_backup_restore_on_cluster/test.py::test_system_functions [gw7] [ 57%] PASSED test_aggregation_memory_efficient/test.py::test_remote test_backup_restore_on_cluster/test.py::test_system_users [gw1] [ 58%] PASSED test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def [gw1] [ 59%] PASSED test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty [gw1] [ 60%] PASSED test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency [gw1] [ 61%] PASSED test_backup_restore_on_cluster/test.py::test_tables_dependency [gw4] [ 62%] ERROR test_disks_app_func/test.py::test_disks_app_func_cp test_disks_app_func/test.py::test_disks_app_func_ld [gw4] [ 63%] ERROR test_disks_app_func/test.py::test_disks_app_func_ld test_disks_app_func/test.py::test_disks_app_func_ln [gw4] [ 64%] ERROR test_disks_app_func/test.py::test_disks_app_func_ln test_disks_app_func/test.py::test_disks_app_func_ls [gw4] [ 65%] ERROR test_disks_app_func/test.py::test_disks_app_func_ls test_disks_app_func/test.py::test_disks_app_func_mv [gw4] [ 66%] ERROR test_disks_app_func/test.py::test_disks_app_func_mv [gw6] [ 67%] PASSED test_detached_parts_metrics/test.py::test_numbers_of_detached_parts [gw2] [ 68%] PASSED test_disk_over_web_server/test.py::test_unavailable_server test_disk_over_web_server/test.py::test_usage[node2] [gw2] [ 69%] PASSED test_disk_over_web_server/test.py::test_usage[node2] [gw9] [ 70%] PASSED test_cgroup_limit/test.py::test_cgroup_cpu_limit [gw0] [ 71%] ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] [gw0] [ 72%] ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk [gw0] [ 73%] ERROR test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries [gw0] [ 74%] ERROR test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works [gw0] [ 75%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] [gw0] [ 76%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] [gw0] [ 77%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] [gw0] [ 78%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] [gw0] [ 79%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] [gw0] [ 80%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] [gw0] [ 81%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] [gw0] [ 82%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] [gw0] [ 83%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] [gw0] [ 84%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] [gw0] [ 85%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] [gw0] [ 86%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] [gw0] [ 87%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] [gw0] [ 88%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] [gw0] [ 89%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] [gw0] [ 90%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] [gw0] [ 91%] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] [gw0] [ 92%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] [gw0] [ 93%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] [gw0] [ 94%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] [gw0] [ 95%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried [gw0] [ 96%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] [gw0] [ 97%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] [gw0] [ 98%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] [gw0] [ 99%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] [gw0] [100%] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] ==================================== ERRORS ==================================== ___________________ ERROR at setup of test_disks_app_func_cp ___________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): global cluster try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "disks_app_test", main_configs=["config.xml"], with_minio=True, ) > cluster.start() test_disks_app_func/test.py:17: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling resolver ... E Pulling minio1 ... E Pulling disks_app_test ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling disks_app_test ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... E Pulling disks_app_test ... status: image is up to date for a... E Pulling disks_app_test ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/cluster.py", line 2831, in start retry(log_function=logging_pulling_images)(run_and_check)( File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 28, in wrapper raise e File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 19, in wrapper func(*args, **kwargs) File "/ClickHouse/tests/integration/helpers/cluster.py", line 129, in run_and_check raise Exception( Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling disks_app_test ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling disks_app_test ... pulling from altinityinfra/integr... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... Pulling disks_app_test ... status: image is up to date for a... Pulling disks_app_test ... done Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied ------------------------------ Captured log setup ------------------------------ 2026-09-30 15:52:20 [ 518 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:52:20 [ 518 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:52:20 [ 518 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-09-30 15:52:20 [ 518 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-09-30 15:52:20 [ 518 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 518 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 518 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV HOSTNAME 383d92be2e22 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV SHLVL 0 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV HOME /root (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV OLDPWD / (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_HELPER_TAG 2cffe1eae894 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PYTHONUNBUFFERED 1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV UBSAN_OPTIONS print_stacktrace=1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PYTEST_ADDOPTS --dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_aggregation_memory_efficient/test.py::test_remote test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True]' test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database]' 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid]' test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False]' 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True]' test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_cgroup_limit/test.py::test_cgroup_cpu_limit 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node]' 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries]' test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded]' test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True]' test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default test_config_xml_yaml_mix/test.py::test_extra_yaml_mix test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_default_database_on_cluster/test.py::test_default_database_on_cluster test_detached_parts_metrics/test.py::test_numbers_of_detached_parts 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed]' test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml test_disabled_access_control_improvements/test_row_policy.py::test_introspection test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies 'test_disk_over_web_server/test.py::test_cache[node2]' test_disk_over_web_server/test.py::test_config_reload test_disk_over_web_server/test.py::test_incorrect_usage test_disk_over_web_server/test.py::test_page_cache test_disk_over_web_server/test.py::test_replicated_database test_disk_over_web_server/test.py::test_unavailable_server 'test_disk_over_web_server/test.py::test_usage[node2]' test_disks_app_func/test.py::test_disks_app_func_cp test_disks_app_func/test.py::test_disks_app_func_ld test_disks_app_func/test.py::test_disks_app_func_ln test_disks_app_func/test.py::test_disks_app_func_ls test_disks_app_func/test.py::test_disks_app_func_mv -vvv (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV COMPOSE_HTTP_TIMEOUT 600 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV CLICKHOUSE_USE_OLD_ANALYZER 1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_KERBERIZED_HADOOP_TAG latest (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_CHANNEL stable (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_CLIENT_TIMEOUT 300 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PWD /ClickHouse/tests/integration (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV TZ Etc/UTC (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-amd64/bin/java (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV DOCKER_BASE_TAG b74d853ebe76 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV LC_CTYPE C.UTF-8 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV INTEGRATION_TESTS_RUN_ID 0 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV WORKER_FREE_PORTS 30200 30201 30202 30203 30204 30205 30206 30207 30208 30209 30210 30211 30212 30213 30214 30215 30216 30217 30218 30219 30220 30221 30222 30223 30224 30225 30226 30227 30228 30229 30230 30231 30232 30233 30234 30235 30236 30237 30238 30239 30240 30241 30242 30243 30244 30245 30246 30247 30248 30249 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PYTEST_XDIST_TESTRUNUID 9d1ef6cf3d754a1298a30637bfa409eb (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PYTEST_XDIST_WORKER gw4 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PYTEST_XDIST_WORKER_COUNT 10 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : ENV PYTEST_CURRENT_TEST test_disks_app_func/test.py::test_disks_app_func_cp (setup) (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : CLUSTER INIT base_config_dir:/clickhouse-config (cluster.py:766, __init__) 2026-09-30 15:52:21 [ 518 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-09-30 15:52:21 [ 518 ] DEBUG : Cluster name: project_name:roottestdisksappfunc_gw4. Added instance name:disks_app_test tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-09-30 15:52:21 [ 518 ] INFO : Running tests in /ClickHouse/tests/integration/test_disks_app_func/test.py (cluster.py:2790, start) 2026-09-30 15:52:21 [ 518 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-09-30 15:52:21 [ 518 ] DEBUG : Docker networks for project roottestdisksappfunc_gw4 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:52:21 [ 518 ] DEBUG : Docker containers for project roottestdisksappfunc_gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:52:21 [ 518 ] DEBUG : Docker volumes for project roottestdisksappfunc_gw4 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:52:21 [ 518 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:52:21 [ 518 ] DEBUG : Docker networks for project roottestdisksappfunc_gw4 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:52:21 [ 518 ] DEBUG : Docker containers for project roottestdisksappfunc_gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:52:21 [ 518 ] DEBUG : Docker volumes for project roottestdisksappfunc_gw4 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:52:21 [ 518 ] DEBUG : Command:docker container list --all --filter name='^/roottestdisksappfunc_gw4_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 518 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:52:21 [ 518 ] DEBUG : No running containers for project: roottestdisksappfunc_gw4 (cluster.py:904, cleanup) 2026-09-30 15:52:21 [ 518 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:52:21 [ 518 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:52:21 [ 518 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 518 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:52:21 [ 518 ] DEBUG : Setup directory for instance: disks_app_test (cluster.py:2810, start) 2026-09-30 15:52:21 [ 518 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_disks_app_func/config.xml'] to /ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/database (cluster.py:4653, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/logs (cluster.py:4664, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4750, create_dir) 2026-09-30 15:52:21 [ 518 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/minio/certs/public.crt'} stored in /ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env (cluster.py:86, _create_env_file) 2026-09-30 15:52:21 [ 518 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:52:21 [ 518 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:52:21 [ 518 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:52:21 [ 518 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:52:21 [ 518 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-09-30 15:52:21 [ 518 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:53:24 [ 518 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:53:24 [ 518 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling minio1 ... Pulling disks_app_test ... Pulling resolver ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling disks_app_test ... pulling from altinityinfra/integr... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... Pulling disks_app_test ... status: image is up to date for a... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling disks_app_test ... done Pulling resolver ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:53:25 [ 518 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... done (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:54:21 [ 518 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:54:21 [ 518 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling disks_app_test ... Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling disks_app_test ... pulling from altinityinfra/integr... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... Pulling disks_app_test ... status: image is up to date for a... Pulling disks_app_test ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:54:24 [ 518 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:55:13 [ 518 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:55:13 [ 518 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling minio1 ... Pulling resolver ... Pulling disks_app_test ... Pulling resolver ... pulling from altinityinfra/python... Pulling disks_app_test ... pulling from altinityinfra/integr... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... Pulling disks_app_test ... status: image is up to date for a... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling disks_app_test ... done Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:55:16 [ 518 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... done (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:56:19 [ 518 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:56:19 [ 518 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling minio1 ... Pulling disks_app_test ... Pulling resolver ... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling resolver ... pulling from altinityinfra/python... Pulling disks_app_test ... pulling from altinityinfra/integr... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling proxy1 ... done Pulling resolver ... done Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... Pulling disks_app_test ... status: image is up to date for a... Pulling disks_app_test ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:56:22 [ 518 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling disks_app_test ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:57:11 [ 518 ] DEBUG : Failed to start cluster: (cluster.py:3178, start) 2026-09-30 15:57:11 [ 518 ] DEBUG : Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling disks_app_test ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling disks_app_test ... pulling from altinityinfra/integr... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... Pulling disks_app_test ... status: image is up to date for a... Pulling disks_app_test ... done Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling minio1 ... error ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:3179, start) 2026-09-30 15:57:11 [ 518 ] DEBUG : None (cluster.py:3180, start) 2026-09-30 15:57:11 [ 518 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:57:11 [ 518 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Docker networks for project roottestdisksappfunc_gw4 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:57:12 [ 518 ] DEBUG : Docker containers for project roottestdisksappfunc_gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:57:12 [ 518 ] DEBUG : Docker volumes for project roottestdisksappfunc_gw4 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:57:12 [ 518 ] DEBUG : Command:docker container list --all --filter name='^/roottestdisksappfunc_gw4_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:57:12 [ 518 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : No running containers for project: roottestdisksappfunc_gw4 (cluster.py:904, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:57:12 [ 518 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:57:12 [ 518 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 15:57:12 [ 518 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Docker networks for project roottestdisksappfunc_gw4 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:57:12 [ 518 ] DEBUG : Docker containers for project roottestdisksappfunc_gw4 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:57:12 [ 518 ] DEBUG : Docker volumes for project roottestdisksappfunc_gw4 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:57:12 [ 518 ] DEBUG : Command:docker container list --all --filter name='^/roottestdisksappfunc_gw4_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:57:12 [ 518 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : No running containers for project: roottestdisksappfunc_gw4 (cluster.py:904, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:57:12 [ 518 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:57:12 [ 518 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) ___________________ ERROR at setup of test_disks_app_func_ld ___________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): global cluster try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "disks_app_test", main_configs=["config.xml"], with_minio=True, ) > cluster.start() test_disks_app_func/test.py:17: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling resolver ... E Pulling minio1 ... E Pulling disks_app_test ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling disks_app_test ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... E Pulling disks_app_test ... status: image is up to date for a... E Pulling disks_app_test ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___________________ ERROR at setup of test_disks_app_func_ln ___________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): global cluster try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "disks_app_test", main_configs=["config.xml"], with_minio=True, ) > cluster.start() test_disks_app_func/test.py:17: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling resolver ... E Pulling minio1 ... E Pulling disks_app_test ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling disks_app_test ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... E Pulling disks_app_test ... status: image is up to date for a... E Pulling disks_app_test ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___________________ ERROR at setup of test_disks_app_func_ls ___________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): global cluster try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "disks_app_test", main_configs=["config.xml"], with_minio=True, ) > cluster.start() test_disks_app_func/test.py:17: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling resolver ... E Pulling minio1 ... E Pulling disks_app_test ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling disks_app_test ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... E Pulling disks_app_test ... status: image is up to date for a... E Pulling disks_app_test ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___________________ ERROR at setup of test_disks_app_func_mv ___________________ [gw4] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def started_cluster(): global cluster try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "disks_app_test", main_configs=["config.xml"], with_minio=True, ) > cluster.start() test_disks_app_func/test.py:17: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... E Pulling proxy1 ... E Pulling resolver ... E Pulling minio1 ... E Pulling disks_app_test ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling disks_app_test ... pulling from altinityinfra/integr... E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling disks_app_test ... digest: sha256:2ea9da862c215fd684... E Pulling disks_app_test ... status: image is up to date for a... E Pulling disks_app_test ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling minio1 ... error E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ________________ ERROR at setup of test_adaptive_timeouts[node] ________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ---------------------------- Captured stdout setup ----------------------------- Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml Copy common default production configuration from /clickhouse-config. Files: config.xml, users.xml ---------------------------- Captured stderr setup ----------------------------- Traceback (most recent call last): File "/ClickHouse/tests/integration/helpers/cluster.py", line 2831, in start retry(log_function=logging_pulling_images)(run_and_check)( File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 28, in wrapper raise e File "/ClickHouse/tests/integration/helpers/retry_decorator.py", line 19, in wrapper func(*args, **kwargs) File "/ClickHouse/tests/integration/helpers/cluster.py", line 129, in run_and_check raise Exception( Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling resolver ... Pulling minio1 ... Pulling node_with_inf_s3_retries ... Pulling node ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling node ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling minio1 ... error Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied ------------------------------ Captured log setup ------------------------------ 2026-09-30 15:52:20 [ 506 ] DEBUG : Command:['docker ps | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:52:20 [ 506 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:52:20 [ 506 ] DEBUG : No running containers (conftest.py:95, cleanup_environment) 2026-09-30 15:52:20 [ 506 ] DEBUG : Pruning Docker networks (conftest.py:97, cleanup_environment) 2026-09-30 15:52:20 [ 506 ] DEBUG : Command:['docker network prune --force'] (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 506 ] DEBUG : Command:["sysctl net.ipv4.ip_local_port_range='55000 65535'"] (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 506 ] DEBUG : Stdout:net.ipv4.ip_local_port_range = 55000 65535 (cluster.py:121, run_and_check) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_KERBEROS_KDC_TAG 9391ecdee8d7 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV CLICKHOUSE_TESTS_SERVER_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV MSAN_OPTIONS abort_on_error=1 poison_in_dtor=1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV JAVA_TOOL_OPTIONS -Djdk.attach.allowAttachSelf=true (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV TSAN_OPTIONS halt_on_error=1 abort_on_error=1 history_size=7 memory_limit_mb=46080 second_deadlock_stack=1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV HOSTNAME 383d92be2e22 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV SHLVL 0 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV HOME /root (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV OLDPWD / (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_HELPER_TAG 2cffe1eae894 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PYTHONUNBUFFERED 1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_PYTHON_BOTTLE_TAG caad4729259e (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV UBSAN_OPTIONS print_stacktrace=1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PYTEST_ADDOPTS --dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_aggregation_memory_efficient/test.py::test_remote test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True]' test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database]' 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid]' test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False]' 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True]' test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_cgroup_limit/test.py::test_cgroup_cpu_limit 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node]' 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries]' test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded]' test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True]' test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default test_config_xml_yaml_mix/test.py::test_extra_yaml_mix test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_default_database_on_cluster/test.py::test_default_database_on_cluster test_detached_parts_metrics/test.py::test_numbers_of_detached_parts 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed]' test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml test_disabled_access_control_improvements/test_row_policy.py::test_introspection test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies 'test_disk_over_web_server/test.py::test_cache[node2]' test_disk_over_web_server/test.py::test_config_reload test_disk_over_web_server/test.py::test_incorrect_usage test_disk_over_web_server/test.py::test_page_cache test_disk_over_web_server/test.py::test_replicated_database test_disk_over_web_server/test.py::test_unavailable_server 'test_disk_over_web_server/test.py::test_usage[node2]' test_disks_app_func/test.py::test_disks_app_func_cp test_disks_app_func/test.py::test_disks_app_func_ld test_disks_app_func/test.py::test_disks_app_func_ln test_disks_app_func/test.py::test_disks_app_func_ls test_disks_app_func/test.py::test_disks_app_func_mv -vvv (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV CLICKHOUSE_LIBRARY_BRIDGE_BINARY_PATH /clickhouse-library-bridge (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV COMPOSE_HTTP_TIMEOUT 600 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_MYSQL_PHP_CLIENT_TAG 88be89c1e3b6 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_DOTNET_CLIENT_TAG 11de0b29a15d (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV CLICKHOUSE_TESTS_CLIENT_BIN_PATH /clickhouse (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_MYSQL_JS_CLIENT_TAG 41ba7c2ec2a1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV CLICKHOUSE_USE_OLD_ANALYZER 1 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PATH /spark-3.3.2-bin-hadoop3/bin:/opt/gdb/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_KERBERIZED_HADOOP_TAG latest (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_CHANNEL stable (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_CLIENT_TIMEOUT 300 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_POSTGRESQL_JAVA_CLIENT_TAG a4eff5c7f4d6 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_NGINX_DAV_TAG b55ac9cd7519 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_MYSQL_GOLANG_CLIENT_TAG 9bec2a638e6e (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PWD /ClickHouse/tests/integration (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_MYSQL_JAVA_CLIENT_TAG 766bff31cfe4 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV CLICKHOUSE_ODBC_BRIDGE_BINARY_PATH /clickhouse-odbc-bridge (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV CLICKHOUSE_TESTS_BASE_CONFIG_DIR /clickhouse-config (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV TZ Etc/UTC (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV JAVA_PATH /usr/lib/jvm/java-11-openjdk-amd64/bin/java (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV DOCKER_BASE_TAG b74d853ebe76 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV SPARK_HOME /spark-3.3.2-bin-hadoop3 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV LC_CTYPE C.UTF-8 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV INTEGRATION_TESTS_RUN_ID 0 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV WORKER_FREE_PORTS 30000 30001 30002 30003 30004 30005 30006 30007 30008 30009 30010 30011 30012 30013 30014 30015 30016 30017 30018 30019 30020 30021 30022 30023 30024 30025 30026 30027 30028 30029 30030 30031 30032 30033 30034 30035 30036 30037 30038 30039 30040 30041 30042 30043 30044 30045 30046 30047 30048 30049 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PYTEST_XDIST_TESTRUNUID 9d1ef6cf3d754a1298a30637bfa409eb (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PYTEST_XDIST_WORKER gw0 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PYTEST_XDIST_WORKER_COUNT 10 (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : ENV PYTEST_CURRENT_TEST test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] (setup) (cluster.py:446, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : CLUSTER INIT base_config_dir:/clickhouse-config (cluster.py:766, __init__) 2026-09-30 15:52:21 [ 506 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-09-30 15:52:21 [ 506 ] DEBUG : Cluster name: project_name:roottestcheckings3blobsparanoid_gw0. Added instance name:node tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-09-30 15:52:21 [ 506 ] DEBUG : clickhouse_start_command: clickhouse server --config-file=/etc/clickhouse-server/{main_config_file} --log-file=/var/log/clickhouse-server/clickhouse-server.log --errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log (cluster.py:1787, add_instance) 2026-09-30 15:52:21 [ 506 ] DEBUG : Cluster name: project_name:roottestcheckings3blobsparanoid_gw0. Added instance name:node_with_inf_s3_retries tag:b74d853ebe76 base_cmd:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml'] docker_compose_yml_dir:/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/ (cluster.py:2081, add_instance) 2026-09-30 15:52:21 [ 506 ] INFO : Starting cluster... (test.py:41, cluster) 2026-09-30 15:52:21 [ 506 ] INFO : Running tests in /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/test.py (cluster.py:2790, start) 2026-09-30 15:52:21 [ 506 ] DEBUG : Cluster start called. is_up=False (cluster.py:2797, start) 2026-09-30 15:52:21 [ 506 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:52:21 [ 506 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:52:21 [ 506 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:52:21 [ 506 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 15:52:21 [ 506 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 15:52:21 [ 506 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 15:52:21 [ 506 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 15:52:21 [ 506 ] DEBUG : Command:docker container list --all --filter name='^/roottestcheckings3blobsparanoid_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 506 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 15:52:21 [ 506 ] DEBUG : No running containers for project: roottestcheckings3blobsparanoid_gw0 (cluster.py:904, cleanup) 2026-09-30 15:52:21 [ 506 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 15:52:21 [ 506 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 15:52:21 [ 506 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 15:52:21 [ 506 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 15:52:21 [ 506 ] DEBUG : Setup directory for instance: node (cluster.py:2810, start) 2026-09-30 15:52:21 [ 506 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/configs/storage_conf.xml'] to /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/database (cluster.py:4653, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/logs (cluster.py:4664, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4750, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Setup directory for instance: node_with_inf_s3_retries (cluster.py:2810, start) 2026-09-30 15:52:21 [ 506 ] DEBUG : Create directory for configuration generated in this helper (cluster.py:4538, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Create directory for common tests configuration (cluster.py:4543, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Copy common configuration from helpers (cluster.py:4563, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Generate and write macros file (cluster.py:4606, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Copy custom test config files ['/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/configs/storage_conf.xml'] to /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/configs/config.d (cluster.py:4636, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Setup database dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/database (cluster.py:4653, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Setup logs dir /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/logs (cluster.py:4664, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Entrypoint cmd: ["clickhouse", "server", "--config-file=/etc/clickhouse-server/config.xml", "--log-file=/var/log/clickhouse-server/clickhouse-server.log", "--errorlog-file=/var/log/clickhouse-server/clickhouse-server.err.log"] (cluster.py:4750, create_dir) 2026-09-30 15:52:21 [ 506 ] DEBUG : Env {'ASAN_OPTIONS': 'use_sigaltstack=0', 'TSAN_OPTIONS': 'use_sigaltstack=0', 'CLICKHOUSE_WATCHDOG_ENABLE': '0', 'CLICKHOUSE_NATS_TLS_SECURE': '0', 'LLVM_PROFILE_FILE': '/var/lib/clickhouse/server_%h_%p_%m.profraw', 'MINIO_CERTS_DIR': '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/minio/certs', 'MINIO_DATA_DIR': '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/minio/data', 'MINIO_PORT': '9001', 'SSL_CERT_FILE': '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/minio/certs/public.crt'} stored in /ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env (cluster.py:86, _create_env_file) 2026-09-30 15:52:21 [ 506 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:52:21 [ 506 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:52:21 [ 506 ] DEBUG : Trying paths: ['/root/.docker/config.json', '/root/.dockercfg'] (config.py:21, find_config_file) 2026-09-30 15:52:21 [ 506 ] DEBUG : No config file found (config.py:28, find_config_file) 2026-09-30 15:52:21 [ 506 ] DEBUG : http://localhost:None "GET /version HTTP/1.1" 200 825 (connectionpool.py:547, _make_request) 2026-09-30 15:52:21 [ 506 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:53:55 [ 506 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:53:55 [ 506 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling minio1 ... Pulling node_with_inf_s3_retries ... Pulling node ... Pulling resolver ... Pulling resolver ... pulling from altinityinfra/python... Pulling node ... pulling from altinityinfra/integr... Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling node ... done Pulling node_with_inf_s3_retries ... done Pulling proxy2 ... done Pulling minio1 ... error Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:53:58 [ 506 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:55:24 [ 506 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:55:24 [ 506 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling resolver ... Pulling minio1 ... Pulling node_with_inf_s3_retries ... Pulling node ... Pulling resolver ... pulling from altinityinfra/python... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling node ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling minio1 ... error Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:55:27 [ 506 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:57:01 [ 506 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:57:01 [ 506 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling minio1 ... Pulling node ... Pulling node_with_inf_s3_retries ... Pulling resolver ... Pulling resolver ... pulling from altinityinfra/python... Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling node ... pulling from altinityinfra/integr... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling proxy2 ... done Pulling node ... done Pulling minio1 ... error Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:57:04 [ 506 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 15:58:46 [ 506 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 15:58:46 [ 506 ] INFO : Got exception pulling images: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... Pulling proxy1 ... Pulling resolver ... Pulling minio1 ... Pulling node ... Pulling node_with_inf_s3_retries ... Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling resolver ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling resolver ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling minio1 ... error Pulling node ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:2827, logging_pulling_images) 2026-09-30 15:58:50 [ 506 ] DEBUG : Command:['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] (cluster.py:113, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling resolver ... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling minio1 ... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node ... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... pulling from altinityinfra/s3-proxy (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy2 ... done (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling proxy1 ... done (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling resolver ... pulling from altinityinfra/python... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling resolver ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling resolver ... done (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node ... done (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling minio1 ... error (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... status: image is up to date for a... (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:Pulling node_with_inf_s3_retries ... done (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr: (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stderr:pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:123, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Exitcode:1 (cluster.py:125, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Failed to start cluster: (cluster.py:3178, start) 2026-09-30 16:00:34 [ 506 ] DEBUG : Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... Pulling proxy2 ... Pulling resolver ... Pulling minio1 ... Pulling node_with_inf_s3_retries ... Pulling node ... Pulling proxy2 ... pulling from altinityinfra/s3-proxy Pulling proxy1 ... pulling from altinityinfra/s3-proxy Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... Pulling proxy2 ... status: image is up to date for a... Pulling proxy2 ... done Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... Pulling proxy1 ... status: image is up to date for a... Pulling proxy1 ... done Pulling resolver ... pulling from altinityinfra/python... Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... Pulling resolver ... status: image is up to date for a... Pulling resolver ... done Pulling node ... pulling from altinityinfra/integr... Pulling node ... digest: sha256:2ea9da862c215fd684... Pulling node ... status: image is up to date for a... Pulling node ... done Pulling minio1 ... error Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... Pulling node_with_inf_s3_retries ... status: image is up to date for a... Pulling node_with_inf_s3_retries ... done ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied (cluster.py:3179, start) 2026-09-30 16:00:34 [ 506 ] DEBUG : None (cluster.py:3180, start) 2026-09-30 16:00:34 [ 506 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 16:00:34 [ 506 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 16:00:34 [ 506 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 16:00:34 [ 506 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 16:00:34 [ 506 ] DEBUG : Command:docker container list --all --filter name='^/roottestcheckings3blobsparanoid_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : No running containers for project: roottestcheckings3blobsparanoid_gw0 (cluster.py:904, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) 2026-09-30 16:00:34 [ 506 ] WARNING : docker-compose up was not called. Trying to export docker.log for running containers (cluster.py:3246, shutdown) 2026-09-30 16:00:34 [ 506 ] DEBUG : Cleanup called (cluster.py:876, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Docker networks for project roottestcheckings3blobsparanoid_gw0 are NETWORK ID NAME DRIVER SCOPE (cluster.py:855, print_all_docker_pieces) 2026-09-30 16:00:34 [ 506 ] DEBUG : Docker containers for project roottestcheckings3blobsparanoid_gw0 are CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES (cluster.py:863, print_all_docker_pieces) 2026-09-30 16:00:34 [ 506 ] DEBUG : Docker volumes for project roottestcheckings3blobsparanoid_gw0 are DRIVER VOLUME NAME (cluster.py:871, print_all_docker_pieces) 2026-09-30 16:00:34 [ 506 ] DEBUG : Command:docker container list --all --filter name='^/roottestcheckings3blobsparanoid_gw0_.*_1$' --format '{{.ID}}:{{.Names}}' (cluster.py:113, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Unstopped containers: {} (cluster.py:890, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : No running containers for project: roottestcheckings3blobsparanoid_gw0 (cluster.py:904, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Trying to prune unused networks... (cluster.py:910, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Trying to prune unused volumes... (cluster.py:937, cleanup) 2026-09-30 16:00:34 [ 506 ] DEBUG : Command:['docker volume ls | wc -l'] (cluster.py:113, run_and_check) 2026-09-30 16:00:34 [ 506 ] DEBUG : Stdout:1 (cluster.py:121, run_and_check) ______ ERROR at setup of test_adaptive_timeouts[node_with_inf_s3_retries] ______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___________________ ERROR at setup of test_no_key_found_disk ___________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception __________ ERROR at setup of test_query_is_canceled_with_inf_retries ___________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______________ ERROR at setup of test_upload_after_check_works ________________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ______ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[br] ______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[bz2] ______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[deflate] ____ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[gzip] _____ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[lz4] ______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[none] _____ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ______ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[xz] ______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _____ ERROR at setup of test_upload_s3_fail_create_multi_part_upload[zstd] _____ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[br] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[none] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[xz] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______ ERROR at setup of test_when_error_is_retried[connection_refused] _______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______ ERROR at setup of test_when_error_is_retried[qps_limit_exceeded] _______ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ___________ ERROR at setup of test_when_error_is_retried[slow_down] ____________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ____ ERROR at setup of test_when_error_is_retried[total_qps_limit_exceeded] ____ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _______ ERROR at setup of test_when_s3_broken_pipe_at_upload_is_retried ________ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception _ ERROR at setup of test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] _ [gw0] linux -- Python 3.10.12 /usr/bin/python3 @pytest.fixture(scope="module") def cluster(): try: cluster = ClickHouseCluster(__file__) cluster.add_instance( "node", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/s3_retries.xml", ], with_minio=True, ) cluster.add_instance( "node_with_inf_s3_retries", main_configs=[ "configs/storage_conf.xml", ], user_configs=[ "configs/setting.xml", "configs/inf_s3_retries.xml", ], with_minio=True, ) logging.info("Starting cluster...") > cluster.start() test_checking_s3_blobs_paranoid/test.py:42: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ helpers/cluster.py:2831: in start retry(log_function=logging_pulling_images)(run_and_check)( helpers/retry_decorator.py:28: in wrapper raise e helpers/retry_decorator.py:19: in wrapper func(*args, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', ...] env = None, shell = False, stdout = -1, stderr = -1, timeout = 900 nothrow = False, detach = False def run_and_check( args, env=None, shell=False, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=300, nothrow=False, detach=False, ): if detach: subprocess.Popen( args, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env, shell=shell, ) return logging.debug(f"Command:{args}") res = subprocess.run( args, stdout=stdout, stderr=stderr, env=env, shell=shell, timeout=timeout ) out = res.stdout.decode("utf-8", "ignore") err = res.stderr.decode("utf-8", "ignore") # check_call(...) from subprocess does not print stderr, so we do it manually for outline in out.splitlines(): logging.debug(f"Stdout:{outline}") for errline in err.splitlines(): logging.debug(f"Stderr:{errline}") if res.returncode != 0: logging.debug(f"Exitcode:{res.returncode}") if env: logging.debug(f"Env:{env}") if not nothrow: > raise Exception( f"Command {args} return non-zero code {res.returncode}: {res.stderr.decode('utf-8')}" ) E Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... E Pulling proxy2 ... E Pulling resolver ... E Pulling minio1 ... E Pulling node_with_inf_s3_retries ... E Pulling node ... E Pulling proxy2 ... pulling from altinityinfra/s3-proxy E Pulling proxy1 ... pulling from altinityinfra/s3-proxy E Pulling proxy2 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy2 ... status: image is up to date for a... E Pulling proxy2 ... done E Pulling proxy1 ... digest: sha256:2c143e57962bb1494a... E Pulling proxy1 ... status: image is up to date for a... E Pulling proxy1 ... done E Pulling resolver ... pulling from altinityinfra/python... E Pulling resolver ... digest: sha256:919cb0e6c1a4f05e8d... E Pulling resolver ... status: image is up to date for a... E Pulling resolver ... done E Pulling node ... pulling from altinityinfra/integr... E Pulling node ... digest: sha256:2ea9da862c215fd684... E Pulling node ... status: image is up to date for a... E Pulling node ... done E Pulling minio1 ... error E Pulling node_with_inf_s3_retries ... pulling from altinityinfra/integr... E Pulling node_with_inf_s3_retries ... digest: sha256:2ea9da862c215fd684... E Pulling node_with_inf_s3_retries ... status: image is up to date for a... E Pulling node_with_inf_s3_retries ... done E E ERROR: for minio1 pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied E pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied helpers/cluster.py:129: Exception ============================== slowest durations =============================== 493.47s setup test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] 291.42s setup test_disks_app_func/test.py::test_disks_app_func_cp 233.57s setup test_disk_over_web_server/test.py::test_cache[node2] 207.26s setup test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible 146.80s setup test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container 128.44s setup test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability 104.85s setup test_ddl_alter_query/test.py::test_alter 103.37s setup test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] 96.65s setup test_default_database_on_cluster/test.py::test_default_database_on_cluster 96.12s setup test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts 91.53s call test_cgroup_limit/test.py::test_cgroup_cpu_limit 60.47s setup test_cleanup_after_start/test.py::test_old_dirs_cleanup 56.28s setup test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] 55.65s setup test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection 55.38s setup test_buffer_profile/test.py::test_buffer_profile 54.65s call test_config_xml_yaml_mix/test.py::test_extra_yaml_mix 54.54s setup test_detached_parts_metrics/test.py::test_numbers_of_detached_parts 51.35s call test_disk_over_web_server/test.py::test_unavailable_server 51.21s setup test_aggregation_memory_efficient/test.py::test_remote 42.60s setup test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async 37.75s setup test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default 37.08s call test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts 35.29s call test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] 31.22s call test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] 18.00s call test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] 15.28s call test_disk_over_web_server/test.py::test_cache[node2] 13.90s call test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] 11.40s teardown test_ddl_alter_query/test.py::test_ddl_queue_hostname_change 10.87s call test_disk_over_web_server/test.py::test_usage[node2] 10.20s call test_disk_over_web_server/test.py::test_page_cache 10.14s teardown test_backup_restore_on_cluster/test.py::test_tables_dependency 9.25s teardown test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async 8.09s call test_detached_parts_metrics/test.py::test_numbers_of_detached_parts 7.93s teardown test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability 6.96s call test_backup_restore_on_cluster/test.py::test_tables_dependency 6.88s teardown test_disk_over_web_server/test.py::test_usage[node2] 6.85s call test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] 6.66s call test_backup_restore_on_cluster/test.py::test_replicated_database_async 6.64s teardown test_default_database_on_cluster/test.py::test_default_database_on_cluster 6.15s call test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection 6.09s call test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async 6.04s call test_backup_restore_on_cluster/test.py::test_required_privileges 6.04s call test_cleanup_after_start/test.py::test_old_dirs_cleanup 5.92s teardown test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container 5.85s teardown test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] 5.57s call test_backup_restore_on_cluster/test.py::test_mutation 5.48s teardown test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies 5.43s teardown test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default 5.30s teardown test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts 5.16s call test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 5.09s call test_backup_restore_on_cluster/test.py::test_replicated_database 4.80s teardown test_buffer_profile/test.py::test_default_profile 4.52s teardown test_detached_parts_metrics/test.py::test_numbers_of_detached_parts 4.50s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def 4.46s call test_backup_restore_on_cluster/test.py::test_keeper_value_max_size 4.33s teardown test_aggregation_memory_efficient/test.py::test_remote 4.30s call test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies 4.20s call test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def 4.02s call test_backup_restore_on_cluster/test.py::test_replicated_table 4.02s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert 3.96s call test_ddl_alter_query/test.py::test_ddl_queue_hostname_change 3.90s call test_backup_restore_on_cluster/test.py::test_system_functions 3.87s call test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] 3.84s call test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica 3.83s teardown test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible 3.81s call test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] 3.76s call test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] 3.71s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge 3.70s call test_backup_restore_on_cluster/test.py::test_system_users 3.57s call test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management 3.52s call test_default_database_on_cluster/test.py::test_default_database_on_cluster 3.49s call test_backup_restore_on_cluster/test.py::test_projection 3.45s call test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath 3.36s call test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] 3.26s teardown test_cleanup_after_start/test.py::test_old_dirs_cleanup 3.04s call test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability 2.98s call test_backup_restore_on_cluster/test.py::test_file_deduplication 2.96s call test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster 2.96s call test_ddl_alter_query/test.py::test_alter 2.92s call test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] 2.45s call test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes 2.43s call test_aggregation_memory_efficient/test.py::test_remote 2.40s call test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty 2.40s call test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container 2.31s call test_backup_restore_on_cluster/test.py::test_empty_replicated_table 2.21s call test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster 2.08s call test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container 2.08s call test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async 2.03s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath 2.03s call test_disk_over_web_server/test.py::test_replicated_database 1.73s teardown test_backup_restore_on_cluster/test.py::test_projection 1.70s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] 1.68s teardown test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty 1.68s teardown test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica 1.68s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 1.68s teardown test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] 1.68s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_async 1.65s call test_disk_over_web_server/test.py::test_incorrect_usage 1.64s teardown test_backup_restore_on_cluster/test.py::test_required_privileges 1.63s teardown test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def 1.63s teardown test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] 1.63s teardown test_backup_restore_on_cluster/test.py::test_file_deduplication 1.63s teardown test_backup_restore_on_cluster/test.py::test_empty_replicated_table 1.63s teardown test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] 1.63s teardown test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] 1.59s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def 1.58s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster 1.58s teardown test_backup_restore_on_cluster/test.py::test_replicated_database 1.55s teardown test_backup_restore_on_cluster/test.py::test_system_functions 1.54s teardown test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] 1.53s teardown test_backup_restore_on_cluster/test.py::test_system_users 1.53s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert 1.53s teardown test_backup_restore_on_cluster/test.py::test_mutation 1.53s teardown test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes 1.53s teardown test_backup_restore_on_cluster/test.py::test_get_error_from_other_host 1.48s teardown test_backup_restore_on_cluster/test.py::test_keeper_value_max_size 1.48s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster 1.48s teardown test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] 1.43s teardown test_backup_restore_on_cluster/test.py::test_replicated_table 1.43s teardown test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge 1.31s call test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default 1.08s call test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml 0.75s call test_disk_over_web_server/test.py::test_config_reload 0.70s call test_backup_restore_on_cluster/test.py::test_get_error_from_other_host 0.59s teardown test_disabled_access_control_improvements/test_row_policy.py::test_introspection 0.58s teardown test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management 0.57s teardown test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection 0.57s teardown test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml 0.53s call test_buffer_profile/test.py::test_buffer_profile 0.53s call test_buffer_profile/test.py::test_default_profile 0.43s call test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible 0.43s call test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible 0.22s call test_disabled_access_control_improvements/test_row_policy.py::test_introspection 0.00s setup test_cgroup_limit/test.py::test_cgroup_cpu_limit 0.00s teardown test_config_xml_yaml_mix/test.py::test_extra_yaml_mix 0.00s teardown test_disk_over_web_server/test.py::test_cache[node2] 0.00s setup test_disk_over_web_server/test.py::test_config_reload 0.00s setup test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] 0.00s setup test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] 0.00s setup test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty 0.00s setup test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 0.00s setup test_backup_restore_on_cluster/test.py::test_get_error_from_other_host 0.00s setup test_config_xml_yaml_mix/test.py::test_extra_yaml_mix 0.00s setup test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] 0.00s setup test_disabled_access_control_improvements/test_row_policy.py::test_introspection 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database 0.00s teardown test_cgroup_limit/test.py::test_cgroup_cpu_limit 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] 0.00s setup test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster 0.00s setup test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies 0.00s setup test_buffer_profile/test.py::test_default_profile 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster 0.00s setup test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] 0.00s teardown test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container 0.00s setup test_backup_restore_on_cluster/test.py::test_system_functions 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] 0.00s setup test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] 0.00s setup test_backup_restore_on_cluster/test.py::test_tables_dependency 0.00s setup test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management 0.00s setup test_backup_restore_on_cluster/test.py::test_mutation 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] 0.00s setup test_backup_restore_on_cluster/test.py::test_projection 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_database_async 0.00s setup test_disk_over_web_server/test.py::test_usage[node2] 0.00s setup test_backup_restore_on_cluster/test.py::test_required_privileges 0.00s setup test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def 0.00s teardown test_ddl_alter_query/test.py::test_alter 0.00s teardown test_disks_app_func/test.py::test_disks_app_func_cp 0.00s setup test_disk_over_web_server/test.py::test_replicated_database 0.00s setup test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table 0.00s teardown test_buffer_profile/test.py::test_buffer_profile 0.00s setup test_backup_restore_on_cluster/test.py::test_system_users 0.00s setup test_disk_over_web_server/test.py::test_page_cache 0.00s teardown test_disk_over_web_server/test.py::test_unavailable_server 0.00s setup test_backup_restore_on_cluster/test.py::test_empty_replicated_table 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] 0.00s setup test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert 0.00s teardown test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible 0.00s setup test_disk_over_web_server/test.py::test_unavailable_server 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 0.00s teardown test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] 0.00s teardown test_disk_over_web_server/test.py::test_page_cache 0.00s teardown test_disk_over_web_server/test.py::test_incorrect_usage 0.00s setup test_backup_restore_on_cluster/test.py::test_file_deduplication 0.00s teardown test_disks_app_func/test.py::test_disks_app_func_mv 0.00s setup test_backup_restore_on_cluster/test.py::test_keeper_value_max_size 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] 0.00s setup test_disks_app_func/test.py::test_disks_app_func_ld 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] 0.00s teardown test_disk_over_web_server/test.py::test_replicated_database 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] 0.00s setup test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] 0.00s setup test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] 0.00s setup test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] 0.00s setup test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] 0.00s setup test_ddl_alter_query/test.py::test_ddl_queue_hostname_change 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] 0.00s teardown test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] 0.00s teardown test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] 0.00s teardown test_disks_app_func/test.py::test_disks_app_func_ld 0.00s setup test_disk_over_web_server/test.py::test_incorrect_usage 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] 0.00s teardown test_disks_app_func/test.py::test_disks_app_func_ls 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] 0.00s setup test_disks_app_func/test.py::test_disks_app_func_ln 0.00s teardown test_disk_over_web_server/test.py::test_config_reload 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] 0.00s teardown test_disks_app_func/test.py::test_disks_app_func_ln 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk 0.00s setup test_disks_app_func/test.py::test_disks_app_func_ls 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] 0.00s setup test_disks_app_func/test.py::test_disks_app_func_mv 0.00s setup test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] =========================== short test summary info ============================ ERROR test_disks_app_func/test.py::test_disks_app_func_cp - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... ERROR test_disks_app_func/test.py::test_disks_app_func_ld - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... ERROR test_disks_app_func/test.py::test_disks_app_func_ln - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... ERROR test_disks_app_func/test.py::test_disks_app_func_ls - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... ERROR test_disks_app_func/test.py::test_disks_app_func_mv - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/.env', '--project-name', 'roottestdisksappfunc_gw4', '--file', '/ClickHouse/tests/integration/test_disks_app_func/_instances_0_gw4/disks_app_test/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', 'pull'] return non-zero code 1: Pulling proxy2 ... ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node] ERROR test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries] ERROR test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk - Exception: Command ['docker-compose', '--env-file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/.env', '--project-name', 'roottestcheckings3blobsparanoid_gw0', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node/docker-compose.yml', '--file', '/ClickHouse/tests/integration/helpers/../../../tests/integration/compose/docker_compose_minio.yml', '--file', '/ClickHouse/tests/integration/test_checking_s3_blobs_paranoid/_instances_0_gw0/node_with_inf_s3_retries/docker-compose.yml', 'pull'] return non-zero code 1: Pulling proxy1 ... ERROR test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz] ERROR test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False] ERROR test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True] PASSED test_buffer_profile/test.py::test_buffer_profile PASSED test_buffer_profile/test.py::test_default_profile PASSED test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection PASSED test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management PASSED test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml PASSED test_disabled_access_control_improvements/test_row_policy.py::test_introspection PASSED test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed] PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed] PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False] PASSED test_ddl_alter_query/test.py::test_alter PASSED test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True] PASSED test_ddl_alter_query/test.py::test_ddl_queue_hostname_change PASSED test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True] PASSED test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica PASSED test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes PASSED test_backup_restore_on_cluster/test.py::test_empty_replicated_table PASSED test_backup_restore_on_cluster/test.py::test_file_deduplication PASSED test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed] PASSED test_backup_restore_on_cluster/test.py::test_get_error_from_other_host PASSED test_backup_restore_on_cluster/test.py::test_keeper_value_max_size PASSED test_cleanup_after_start/test.py::test_old_dirs_cleanup PASSED test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container PASSED test_backup_restore_on_cluster/test.py::test_mutation PASSED test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async PASSED test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container PASSED test_backup_restore_on_cluster/test.py::test_projection PASSED test_backup_restore_on_cluster/test.py::test_replicated_database PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_async PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database] PASSED test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default PASSED test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid] PASSED test_backup_restore_on_cluster/test.py::test_replicated_table PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster PASSED test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def PASSED test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge PASSED test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath PASSED test_backup_restore_on_cluster/test.py::test_required_privileges PASSED test_default_database_on_cluster/test.py::test_default_database_on_cluster PASSED test_config_xml_yaml_mix/test.py::test_extra_yaml_mix PASSED test_disk_over_web_server/test.py::test_cache[node2] PASSED test_disk_over_web_server/test.py::test_config_reload PASSED test_disk_over_web_server/test.py::test_incorrect_usage PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False] PASSED test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts PASSED test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability PASSED test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True] PASSED test_disk_over_web_server/test.py::test_page_cache PASSED test_disk_over_web_server/test.py::test_replicated_database PASSED test_backup_restore_on_cluster/test.py::test_system_functions PASSED test_aggregation_memory_efficient/test.py::test_remote PASSED test_backup_restore_on_cluster/test.py::test_system_users PASSED test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def PASSED test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty PASSED test_backup_restore_on_cluster/test.py::test_tables_dependency PASSED test_detached_parts_metrics/test.py::test_numbers_of_detached_parts PASSED test_disk_over_web_server/test.py::test_unavailable_server PASSED test_disk_over_web_server/test.py::test_usage[node2] PASSED test_cgroup_limit/test.py::test_cgroup_cpu_limit ================== 65 passed, 35 errors in 498.01s (0:08:18) =================== Traceback (most recent call last): File "/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration/./runner", line 445, in subprocess.check_call(cmd, shell=True) File "/usr/lib/python3.10/subprocess.py", line 369, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command 'docker run --rm --name clickhouse_integration_tests_fuqawq --privileged --dns-search='.' --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-odbc-bridge:/clickhouse-odbc-bridge --volume=/home/ubuntu/_work/_temp/test/build/clickhouse:/clickhouse --volume=/home/ubuntu/_work/_temp/test/build/clickhouse-library-bridge:/clickhouse-library-bridge --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/programs/server:/clickhouse-config --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/tests/integration:/ClickHouse/tests/integration --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/backupview:/ClickHouse/utils/backupview --volume=/home/ubuntu/_work/ClickHouse/ClickHouse/utils/grpc-client/pb2:/ClickHouse/utils/grpc-client/pb2 --volume=/run:/run/host:ro --volume=clickhouse_integration_tests_volume:/var/lib/docker -e DOCKER_DOTNET_CLIENT_TAG=11de0b29a15d -e DOCKER_HELPER_TAG=2cffe1eae894 -e DOCKER_BASE_TAG=b74d853ebe76 -e DOCKER_KERBEROS_KDC_TAG=9391ecdee8d7 -e DOCKER_MYSQL_GOLANG_CLIENT_TAG=9bec2a638e6e -e DOCKER_MYSQL_JAVA_CLIENT_TAG=766bff31cfe4 -e DOCKER_MYSQL_JS_CLIENT_TAG=41ba7c2ec2a1 -e DOCKER_MYSQL_PHP_CLIENT_TAG=88be89c1e3b6 -e DOCKER_NGINX_DAV_TAG=b55ac9cd7519 -e DOCKER_POSTGRESQL_JAVA_CLIENT_TAG=a4eff5c7f4d6 -e DOCKER_PYTHON_BOTTLE_TAG=caad4729259e -e DOCKER_CLIENT_TIMEOUT=300 -e COMPOSE_HTTP_TIMEOUT=600 -e CLICKHOUSE_USE_OLD_ANALYZER=1 -e PYTHONUNBUFFERED=1 -e PYTEST_ADDOPTS="--dist=loadfile -n 10 -rfEps --run-id=0 --color=no --durations=0 test_aggregation_memory_efficient/test.py::test_remote test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_different_container test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_native_copy_async test_azure_blob_storage_native_copy/test.py::test_backup_restore_on_merge_tree_same_container 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-False]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[http-True]' 'test_backup_restore_on_cluster/test.py::test_async_backups_to_same_destination[native-True]' test_backup_restore_on_cluster/test.py::test_backup_restore_on_single_replica test_backup_restore_on_cluster/test.py::test_different_tables_on_nodes test_backup_restore_on_cluster/test.py::test_empty_replicated_table test_backup_restore_on_cluster/test.py::test_file_deduplication test_backup_restore_on_cluster/test.py::test_get_error_from_other_host test_backup_restore_on_cluster/test.py::test_keeper_value_max_size test_backup_restore_on_cluster/test.py::test_mutation test_backup_restore_on_cluster/test.py::test_projection test_backup_restore_on_cluster/test.py::test_replicated_database test_backup_restore_on_cluster/test.py::test_replicated_database_async test_backup_restore_on_cluster/test.py::test_replicated_database_compare_parts 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[database]' 'test_backup_restore_on_cluster/test.py::test_replicated_database_with_special_macro_in_zk_path[uuid]' test_backup_restore_on_cluster/test.py::test_replicated_table test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_bigger_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_restored_into_smaller_cluster test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_def test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_insert test_backup_restore_on_cluster/test.py::test_replicated_table_with_not_synced_merge test_backup_restore_on_cluster/test.py::test_replicated_table_with_uuid_in_zkpath test_backup_restore_on_cluster/test.py::test_required_privileges 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[False]' 'test_backup_restore_on_cluster/test.py::test_stop_other_host_during_backup[True]' test_backup_restore_on_cluster/test.py::test_system_functions test_backup_restore_on_cluster/test.py::test_system_users test_backup_restore_on_cluster/test.py::test_table_in_replicated_database_with_not_synced_def test_backup_restore_on_cluster/test.py::test_table_with_parts_in_queue_considered_non_empty test_backup_restore_on_cluster/test.py::test_tables_dependency test_backup_restore_on_cluster/test_slow_rmt.py::test_replicated_database_async test_backward_compatibility/test_insert_profile_events.py::test_new_client_compatible test_backward_compatibility/test_insert_profile_events.py::test_old_client_compatible test_backward_compatibility/test_parallel_replicas_protocol.py::test_backward_compatability test_backward_compatibility/test_vertical_merges_from_compact_parts.py::test_vertical_merges_from_compact_parts test_buffer_profile/test.py::test_buffer_profile test_buffer_profile/test.py::test_default_profile test_cgroup_limit/test.py::test_cgroup_cpu_limit 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node]' 'test_checking_s3_blobs_paranoid/test.py::test_adaptive_timeouts[node_with_inf_s3_retries]' test_checking_s3_blobs_paranoid/test.py::test_no_key_found_disk test_checking_s3_blobs_paranoid/test.py::test_query_is_canceled_with_inf_retries test_checking_s3_blobs_paranoid/test.py::test_upload_after_check_works 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_create_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[br]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[bz2]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[deflate]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[gzip]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[lz4]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[none]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[xz]' 'test_checking_s3_blobs_paranoid/test.py::test_upload_s3_fail_upload_part_when_multi_part_upload[zstd]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[connection_refused]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[qps_limit_exceeded]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[slow_down]' 'test_checking_s3_blobs_paranoid/test.py::test_when_error_is_retried[total_qps_limit_exceeded]' test_checking_s3_blobs_paranoid/test.py::test_when_s3_broken_pipe_at_upload_is_retried 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_create_mpu_retried[True]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[False]' 'test_checking_s3_blobs_paranoid/test.py::test_when_s3_connection_reset_by_peer_at_upload_is_retried[True]' test_cleanup_after_start/test.py::test_old_dirs_cleanup test_concurrent_threads_soft_limit/test.py::test_concurrent_threads_soft_limit_default test_config_xml_yaml_mix/test.py::test_extra_yaml_mix test_ddl_alter_query/test.py::test_alter test_ddl_alter_query/test.py::test_ddl_queue_hostname_change test_default_database_on_cluster/test.py::test_default_database_on_cluster test_detached_parts_metrics/test.py::test_numbers_of_detached_parts 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_complex[complex_key_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_ranged[range_hashed]' 'test_dictionaries_all_layouts_separate_sources/test_executable_hashed.py::test_simple[hashed]' test_disabled_access_control_improvements/test_row_policy.py::test_dcl_introspection test_disabled_access_control_improvements/test_row_policy.py::test_dcl_management test_disabled_access_control_improvements/test_row_policy.py::test_dcl_users_with_policies_from_users_xml test_disabled_access_control_improvements/test_row_policy.py::test_introspection test_disabled_access_control_improvements/test_row_policy.py::test_some_users_without_policies 'test_disk_over_web_server/test.py::test_cache[node2]' test_disk_over_web_server/test.py::test_config_reload test_disk_over_web_server/test.py::test_incorrect_usage test_disk_over_web_server/test.py::test_page_cache test_disk_over_web_server/test.py::test_replicated_database test_disk_over_web_server/test.py::test_unavailable_server 'test_disk_over_web_server/test.py::test_usage[node2]' test_disks_app_func/test.py::test_disks_app_func_cp test_disks_app_func/test.py::test_disks_app_func_ld test_disks_app_func/test.py::test_disks_app_func_ln test_disks_app_func/test.py::test_disks_app_func_ls test_disks_app_func/test.py::test_disks_app_func_mv -vvv" altinityinfra/integration-tests-runner:cf95077f4609 ' returned non-zero exit status 1.